Information processing apparatus, information processing method and program

ABSTRACT

There is provided an information processing apparatus including a matter extracting unit extracting a predetermined matter from text information, an action pattern specifying unit specifying one or multiple action patterns associated with the predetermined matter, an action extracting unit extracting each of the action patterns associated with the predetermined matter, from sensor information, and a state analyzing unit generating state information indicating a state related to the matter, based on each of the action patterns extracted from the sensor information, using a contribution level indicating a degree of contribution of each of the action patterns to the predetermined matter, for a combination of the predetermined matter and each of the action patterns associated with the predetermined matter.

CROSS-REFERENCE OF RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/453,508, filed Mar. 8, 2017, now patented as U.S. Pat. No. 9,978,259,which is a continuation of U.S. application Ser. No. 14/613,825, filedFeb. 4, 2015, now patented as U.S. Pat. No. 9,619,998, which is acontinuation of U.S. application Ser. No. 13/858,297, filed Apr. 8,2013, now patented as U.S. Pat. No. 8,981,941, which is based upon andclaims benefit of priority from Prior Japanese Patent Application No.2012-126051, filed Jun. 1, 2012, the entire contents of all of which areincorporated herein by reference.

BACKGROUND

The present disclosure relates to an information processing apparatus,an information processing method and a program.

It is focused on a technique of mounting a motion sensor on a mobileterminal such as a mobile phone and automatically detecting andrecording a use action history. For example, following Japanese PatentLaid-Open No. 2008-003655 discloses a technique of using a motion sensorsuch as an acceleration sensor and a gyro sensor and detecting a walkingoperation, a running operation, a right-turning and left-turningoperation and a still state. The patent literature discloses a method ofcalculating a walking pitch, walking power and a rotation angle fromoutput data of the motion sensor and detecting the walking operation,the running operation, the right-turning and left-turning operation andthe still state using the calculation result.

Further, the patent literature discloses a method of detecting a user'saction pattern by statistical processing with an input of operation andstate patterns such as the types of these operations and state, theperiod of time during which the operations and the state continue andthe number of operations. By using the above method, it is possible toacquire an action pattern such as “sauntering” and “restless operation”as time-series data. However, the action pattern acquired in this methodmainly indicates a user's operation and state performed in a relativelyshort period of time. Therefore, it is difficult to estimate, from anaction pattern history, specific action content such as “I shopped at adepartment store today” and “I ate at a hotel restaurant yesterday.”

The action pattern acquired using the method disclosed in followingJapanese Patent Laid-Open No. 2008-003655 denotes an accumulation ofactions performed in a relatively period of time. Also, individualactions themselves forming the action pattern are not intentionallyperformed by the user. By contrast, specific action content isintentionally performed by the user in most cases and is highlyentertaining, which is performed over a relatively long period of time.Therefore, it is difficult to estimate the above specific action contentfrom an accumulation of actions performed during a short period of time.However, recently, there is developed a technique of detecting ahighly-entertaining action pattern performed over a relatively longperiod of time, from an action pattern in a relatively short period oftime acquired using a motion sensor (see following Japanese PatentLaid-Open No. 2011-081431).

SUMMARY

Meanwhile, recently, a network environment surrounding users becomesophisticated and diversified, and social network services have becomecommon, which upload a comment input by the user to a server on anetwork. Such a comment may include information related to a user'saction or intention.

The present disclosure is considered in view of such a condition andintends to provide a new improved information processing apparatus,information processing method and program that can provide higher levelinformation by combining an action pattern recognition result based oninformation acquired from a position sensor or motion sensor and otherinformation than the information acquired form the position sensor ormotion sensor.

According to an embodiment of the present disclosure, there is providedan information processing apparatus including a matter extracting unitextracting a predetermined matter from text information, an actionpattern specifying unit specifying one or multiple action patternsassociated with the predetermined matter, an action extracting unitextracting each of the action patterns associated with the predeterminedmatter, from sensor information, and a state analyzing unit generatingstate information indicating a state related to the predeterminedmatter, based on each of the action patterns extracted from the sensorinformation, using a contribution level indicating a degree ofcontribution of each of the action patterns to the predetermined matter,for a combination of the predetermined matter and each of the actionpatterns associated with the predetermined matter.

According to an embodiment of the present disclosure, there is providedan information processing method including extracting a predeterminedmatter from text information, specifying one or multiple action patternsassociated with the predetermined matter, extracting each of the actionpatterns associated with the predetermined matter, from sensorinformation, and generating state information indicating a state relatedto the predetermined matter, based on each of the action patternsextracted from the sensor information, using a contribution levelindicating a degree of contribution of each of the action patterns tothe predetermined matter, for a combination of the predetermined matterand each of the action patterns associated with the predeterminedmatter.

According to an embodiment of the present disclosure, there is provideda program for causing a computer to realize a matter extracting functionof extracting a predetermined matter from text information, an actionpattern specifying function of specifying one or multiple actionpatterns associated with the predetermined matter, an action extractingfunction of extracting each of the action patterns associated with thepredetermined matter, from sensor information, and a state analyzingfunction of generating state information indicating a state related tothe matter, based on each of the action patterns extracted from thesensor information, using a contribution level indicating a degree ofcontribution of each of the action patterns to the predetermined matter,for a combination of the predetermined matter and each of the actionpatterns associated with the predetermined matter.

According to the embodiments of present disclosure described above, itis possible to provide higher level information by combining an actionpattern recognition result based on information acquired from a positionsensor or motion sensor and other information than the informationacquired form the position sensor or motion sensor.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an explanatory diagram for explaining a configuration exampleof an action/situation analysis system;

FIG. 2 is an explanatory diagram for explaining a function of amotion/state recognizing unit;

FIG. 3 is an explanatory diagram for explaining a function of amotion/state recognizing unit;

FIG. 4 is an explanatory diagram for explaining a function of a GISinformation acquiring unit;

FIG. 5 is an explanatory diagram for explaining a function of a GISinformation acquiring unit;

FIG. 6 is an explanatory diagram for explaining a function of a GISinformation acquiring unit;

FIG. 7 is an explanatory diagram for explaining a function of a GISinformation acquiring unit;

FIG. 8 is an explanatory diagram for explaining a function of anaction/situation recognizing unit;

FIG. 9 is an explanatory diagram for explaining a function of anaction/situation recognizing unit;

FIG. 10 is an explanatory diagram for explaining an action/situationpattern decision method;

FIG. 11 is an explanatory diagram for explaining a calculation method ofscore distribution using a geo histogram;

FIG. 12 is an explanatory diagram for explaining a calculation method ofscore distribution using machine learning;

FIG. 13 is an explanatory diagram for explaining an example of adetected action/situation pattern;

FIG. 14 is an explanatory diagram for explaining an example of a systemconfiguration according to an embodiment of the present disclosure;

FIG. 15 is an explanatory diagram for explaining a configuration of aninformation provision system according to configuration example #1;

FIG. 16 is an explanatory diagram for explaining details of a functionof an information provision system according to configuration example#1;

FIG. 17 is an explanatory diagram for explaining details of a functionof an information provision system according to configuration example#1;

FIG. 18 is an explanatory diagram for explaining details of a functionof an information provision system according to configuration example#1;

FIG. 19 is an explanatory diagram for explaining details of a functionof an information provision system according to configuration example#1;

FIG. 20 is an explanatory diagram for explaining details of a functionof an information provision system according to configuration example#1;

FIG. 21 is an explanatory diagram for explaining details of a functionof an information provision system according to configuration example#1;

FIG. 22 is an explanatory diagram for explaining details of a functionof an information provision system according to configuration example#1;

FIG. 23 is an explanatory diagram for explaining an operation of aninformation provision system according to configuration example #1;

FIG. 24 is an explanatory diagram for explaining a configuration of aninformation provision system according to configuration example #2;

FIG. 25 is an explanatory diagram for explaining details of a functionof an information provision system according to configuration example#2;

FIG. 26 is an explanatory diagram for explaining details of a functionof an information provision system according to configuration example#2;

FIG. 27 is an explanatory diagram for explaining details of a functionof an information provision system according to configuration example#2;

FIG. 28 is an explanatory diagram for explaining an operation of aninformation provision system according to configuration example #2;

FIG. 29 is an explanatory diagram for explaining a configuration of aninformation provision system according to configuration example #3;

FIG. 30 is an explanatory diagram for explaining details of a functionand decision operation of an information provision system according toconfiguration example #3; and

FIG. 31 is an explanatory diagram for explaining a hardwareconfiguration example that can realize the functions of a system andeach device according to the embodiment.

DETAILED DESCRIPTION OF THE EMBODIMENT(S)

Hereinafter, preferred embodiments of the present disclosure will bedescribed in detail with reference to the appended drawings. Note that,in this specification and the appended drawings, structural elementsthat have substantially the same function and structure are denoted withthe same reference numerals, and repeated explanation of thesestructural elements is omitted.

[Regarding Flow of Explanation]

Here, a flow of explanation disclosed herein is simply described.

First, with reference to FIG. 1 to FIG. 13, an action patternrecognition technique related to a technique of the present embodimentis explained. Next, with reference to FIG. 14, an example of a systemconfiguration according to an embodiment of the present disclosure isexplained. Next, with reference to FIG. 15 to FIG. 23, a function andoperation of an information provision system 13 according toconfiguration example #1 are explained.

Next, with reference to FIG. 24 to FIG. 28, a function and operation ofan information provision system 17 according to configuration example #2are explained. Next, with reference to FIG. 29 and FIG. 30, a functionand operation of an information provision system 19 according toconfiguration example #3 are explained. Next, with reference to FIG. 31,a hardware configuration example that can realize the functions of asystem and each device according to the embodiment is explained.

Finally, technical ideas according to the embodiment are summarized andan operational effect acquired from the technical ideas is simplyexplained.

Explanation Items

-   1: Introduction-   1-1: Action pattern recognition technique-   1-2: Outline of embodiment-   2: Details of embodiment-   2-1: Example of system configuration-   2-2: Configuration example #1 (suggestion of goal attainment level)-   2-2-1: Details of system configuration-   2-2-2: Flow of processing-   2-2-3: Example of screen display-   2-2-4: Alternation example (application to animals)-   2-3: Configuration example #2 (display of detailed action)-   2-3-1: Details of system configuration-   2-3-2: Flow of processing-   2-3-3: Example of screen display-   2-4: Configuration example #3: (decision of ordinary action or    extraordinary action)-   2-4-1: Details of system configuration-   2-4-2: Application example-   2-5: Regarding combination of configuration examples-   3: Example Hardware Configuration-   4: Conclusion

1: INTRODUCTION

First, an action pattern recognition technique related to a technique ofthe present embodiment is explained.

[1-1: Action Pattern Recognition Technique]

The action pattern recognition technique explained herein relates to atechnique of detecting a user's action and state using informationrelated to a user's action and state detected by a motion sensor or thelike and position information detected by a position sensor or the like.

Also, as the motion sensor, for example, a triaxial acceleration sensor(including an acceleration sensor, a gravity detection sensor and a falldetection sensor) and a triaxial gyro sensor (including an angularvelocity sensor, a stabilization sensor and a terrestrial magnetismsensor) are used. Also, for example, it is possible to use informationof GPS (Global Positioning System), RFID (Radio FrequencyIdentification), Wi-Fi access points or wireless base stations as theposition sensor. By using their information, for example, it is possibleto detect the latitude and longitude of the current position.

(System Configuration of Action/Situation Analysis System 10)

First, with reference to FIG. 1, an explanation is given to the systemconfiguration of the action/situation analysis system 10 that canrealize the action pattern recognition technique as described above.FIG. 1 is an explanatory diagram for explaining the entire systemconfiguration of the action/situation analysis system 10.

Here, in the present specification, expression “motion/state” andexpression “action/situation” are separated by the following meanings.The expression “motion/state” denotes an action performed by the user ina relatively short period of time of around several seconds to severalminutes, and indicates behavior such as “walking,” “running,” “jumping”and “still.” Also, this behavior may be collectively expressed as“motion/state pattern” or “LC (Low-Context) action.” Meanwhile, theexpression “action/situation” denotes living activities performed by theuser in a longer period of time than that in the case of “motion/state,”and indicates behavior such as “eating,” “shopping” and “working.” Also,this behavior may be collectively expressed as “action/situationpattern” or “HC (High-Context) action.”

As illustrated in FIG. 1, the action/situation analysis system 10 mainlyincludes a motion sensor 101, a motion/state recognizing unit 102, atime information acquiring unit 103, a position sensor 104, a GISinformation acquiring unit 105 and an action/situation recognizing unit106.

Also, the action/situation analysis system 10 may include an applicationAP or service SV using an action/situation pattern detected by theaction/situation recognizing unit 106. Also, it may be formed such thatan action/situation pattern use result by the application AP and userprofile information are input in the action/situation recognizing unit106.

First, when the user acts, the motion sensor 101 detects a change ofacceleration or rotation around the gravity axis (hereinafter referredto as “sensor data”). The sensor data detected by the motion sensor 101is input in the motion/state recognizing unit 102 as illustrated in FIG.2.

When the sensor data is input, as illustrated in FIG. 2, themotion/state recognizing unit 102 detects a motion/state pattern usingthe input sensor data. As illustrated in FIG. 3, examples of themotion/state pattern that can be detected by the motion/staterecognizing unit 102 include “walking,” “running,” “still,” “jumping,”“train (riding/non-riding)” and “elevator(riding/non-riding/rising/falling).” The motion/state pattern detectedby the motion/state recognizing unit 102 is input in theaction/situation recognizing unit 106.

The position sensor 104 continuously or intermittently acquires positioninformation indicating a user's location (hereinafter referred to as“current position”). For example, the position information of thecurrent position is expressed by latitude and longitude. The positioninformation of the current position acquired by the position sensor 104is input in the GIS information acquiring unit 105.

When the position information of the current position is input, the GISinformation acquiring unit 105 acquires GIS (Geographic InformationSystem) information. Subsequently, as illustrated in FIG. 4, the GISinformation acquiring unit 105 detects an attribute of the currentposition using the acquired GIS information. For example, the GISinformation includes map information and various kinds of informationacquired by an artificial satellite or field survey, which isinformation used for scientific research, management of land, facilitiesor road and urban design. When the GIS information is used, it ispossible to decide an attribute of the current position. For example,the GIS information acquiring unit 105 expresses the attribute of thecurrent position using identification information called “geo categorycode” (for example, see FIG. 5).

As illustrated in FIG. 5, the geo category code denotes a classificationcode to classify the type of information related to a place. This geocategory code is set depending on, for example, a construction type, alandform shape, a geological feature, locality, and so on. Therefore, byspecifying the geo category code of the current position, it is possibleto recognize an environment in which the user is placed, in some degree.

The GIS information acquiring unit 105 refers to the acquired GISinformation, specifies a construction or the like in the currentposition and the periphery of the current position, and extracts a geocategory code corresponding to the construction or the like. The geocategory code selected by the GIS information acquiring unit 105 isinput in the action/situation recognizing unit 106. Also, in a casewhere there are many constructions or the like in the periphery of thecurrent position, the GIS information acquiring unit 105 may extract thegeo category code of each construction and input information such as geohistograms illustrated in FIG. 6 and FIG. 7, as information related tothe extracted geo category, in the action/situation recognizing unit106.

As illustrated in FIG. 8, the action/situation recognizing unit 106receives an input of the motion/state pattern from the motion/staterecognizing unit 102 and an input of the geo category code from the GISinformation acquiring unit 105. Also, the action/situation recognizingunit 106 receives an input of time information from the time informationacquiring unit 103. This time information includes informationindicating the time at which the motion sensor 101 acquires the sensordata. Also, this time information may include information indicating thetime at which the position sensor 104 acquires the position information.Also, the time information may include information such as dayinformation, holiday information and date information, in addition tothe information indicating the time.

When the above information is input, the action/situation recognizingunit 106 detects an action/situation pattern based on the inputmotion/state pattern, the input geo category code (or the geohistograms, for example) and the input time information. At this time,the action/situation recognizing unit 106 detects the action/situationpattern using decision processing based on rules (hereinafter referredto as “rule base decision”) and decision processing based on learningmodels (hereinafter referred to as “learning model decision”). In thefollowing, the rule base decision and the learning model decision aresimply explained.

(Regarding Rule Base Decision)

First, the rule base decision is explained. The rule base decisiondenotes a method of assigning scores to combinations of geo categorycodes and action/situation patterns and deciding an appropriateaction/situation pattern corresponding to input data based on thescores.

A score assignment rule is realized by a score map SM as illustrated inFIG. 9. The score map SM is prepared for each time information, such asdate, time zone and day. For example, a score map SM supporting Mondayin the first week of March is prepared. Further, the score map SM isprepared for each motion/state pattern, such as walking, running andtrain. For example, a score map SM during walking is prepared.Therefore, a score map SM is prepared for each of combinations of timeinformation and motion/state patterns.

As illustrated in FIG. 10, the action/situation recognizing unit 106selects a score map SM suitable to input time information andmotion/state pattern, from multiple score maps SM prepared in advance.Also, as illustrated in FIG. 11, the action/situation recognizing unit106 extracts scores corresponding to a geo category code, from theselected score map SM. By this processing, taking into account a stateof the current position at the acquisition time of sensor data, theaction/situation recognizing unit 106 can extract the score of eachaction/situation pattern existing in the score map SM.

Next, the action/situation recognizing unit 106 specifies the maximumscore among the extracted scores and extracts an action/situationpattern corresponding to the maximum score. Thus, a method of detectingan action/situation pattern is the rule base decision. Here, a score inthe score map SM indicates an estimated probability that the user takesan action/situation pattern corresponding to the score. That is, thescore map SM indicates score distribution of action/situation patternsestimated to be taken by the user in a state of the current positionexpressed by a geo category code.

For example, at around three o'clock on Sunday, it is estimated that theuser in a department store is highly likely to be “shopping.” However,at around 19 o'clock in the same department store, it is estimated thatthe user in the department store is highly likely to be “eating.” Thus,in a certain place, score distribution of action/situation patternsperformed by the user denotes the score map SM (accurately, score map SMgroup).

For example, the score map SM may be input in advance by the userhimself/herself or somebody else, or may be acquired using machinelearning or the like. Also, the score map SM may be optimized bypersonal profile information PR or action/situation feedback FB (rightand wrong of output action/situation pattern) acquired from the user. Asthe profile information PR, for example, age, gender, job or homeinformation and workplace information are used. The above is specificprocessing content of the rule base decision.

(Regarding Learning Model Decision)

Next, the learning model decision is explained. The learning modeldecision is a method of generating a decision model to decide anaction/situation pattern by a machine learning algorithm and deciding anaction/situation pattern corresponding to input data by the generateddecision model.

As the machine learning algorithm, for example, a k-men method, anearest neighbor method, SVM, HMM and boosting are available. Here, SVMis an abbreviation of “Support Vector Machine.” Also, HMM is anabbreviation of “Hidden Markov Model.” In addition to these methods,there is a method of generating a decision model using an algorithmconstruction method based on genetic search disclosed in Japanese PatentLaid-Open No. 2009-48266.

As a feature amount vector input in a machine learning algorithm, forexample, as illustrated in FIG. 12, time information, motion/statepattern, geo category code (or geo category histogram), sensor data andposition information of the current position are available. Here, in thecase of using an algorithm construction method based on generic search,a generic search algorithm is used on a feature amount vector selectionstage in learning process. First, the action/situation recognizing unit106 inputs a feature amount vector in which a correct action/situationpattern is known, in a machine learning algorithm, as learning data, andgenerates a decision model to decide the accuracy of eachaction/situation pattern or an optimal action/situation pattern.

Next, the action/situation recognizing unit 106 inputs input data in thegenerated decision model and decides an action/situation patternestimated to be suitable to the input data. However, in a case where itis possible to acquire right and wrong feedback with respect to a resultof decision performed using the generated decision model, the decisionmodel is reconstructed using the feedback. In this case, theaction/situation recognizing unit 106 decides an action/situationpattern estimated to be suitable to the input data using thereconstructed decision model. The above is specific processing contentof the learning model decision.

By the above-described method, the action/situation recognizing unit 106detects an action/situation pattern as illustrated in FIG. 13.Subsequently, the action/situation pattern detected by theaction/situation recognizing unit 106 is used to provide recommendedservice SV based on the action/situation pattern or used by anapplication AP that performs processing based on the action/situationpattern.

The system configuration of the action/situation analysis system 10 hasbeen described above. Techniques according to an embodiment describedbelow relate to functions of the action/situation analysis system 10described above. Also, regarding detailed functions of theaction/situation analysis system 10, for example, the disclosure ofJapanese Patent Laid-Open No. 2011-081431 serves as a reference.

[1-2: Outline of Embodiment]

In the following, an outline of the present embodiment is described.Techniques according to the present embodiment relate to a system ofproviding information of high value by combining action patterninformation acquired by using the above action/situation analysis system10 and input information such as text information.

For example, configuration example #1 introduced below relates to asystem of providing, based on one or multiple action patternscorresponding to a “predetermined matter,” “state information”representing a state related to the matter. For example, the above“predetermined matter” denotes the user's goal/declaration acquired frominput information and the above “state information” denotes theattainment level with respect to the goal/declaration.

The above “predetermined matter” is not limited to the user'sgoal/declaration acquired from input information and the above “stateinformation” is not limited to the attainment level with respect to thegoal/declaration, but, in configuration example #1 described below, anexplanation is given using an example where the attainment level withrespect to the user's goal/declaration mainly acquired from inputinformation is provided to the user.

Also, in addition to the attainment level which is an example ofcomparison information between the current state with respect to apredetermined matter and a state in a case where the goal/declaration isattained, for example, the “state information” may denote informationindicating the current state with respect to the predetermined matter orcomparison information between the current state with respect to thepredetermined matter and a past state. Even in this case, a techniqueaccording to configuration example #1 described below is applicable.

Also, configuration #2 described below relates to a system of: attachinginformation related to a user's experience acquired from inputinformation such as text information to action pattern informationacquired by the above action/situation analysis system 10; and providingmore detailed information to the user. Further, configuration example #3described below relates to a system of: deciding an explanatory actionor experience among action pattern information acquired using the aboveaction/situation analysis system 10 and a user's experience acquiredfrom input information such as text information; and providing it to theuser.

Also, it is possible to arbitrarily combine the techniques according tothese configuration examples #1 to #3. Also, in the followingexplanation, although text information is mainly assumed as inputinformation used for experience extraction, for example, it is possibleto use sound information acquired using a microphone. In this case, itis possible to acquire information related to a surrounding environmentor action using a waveform of the sound signal as is, or it is possibleto acquire text information from the sound signal using a soundrecognition technique. Since it is possible to acquire text informationin the case of using the sound recognition technique, it is possible toapply the techniques according to below-described configuration examples#1 to #3 as is.

2: DETAILS OF EMBODIMENT

In the following, details of techniques according to the presentembodiment are explained.

[2-1: Example of System Configuration]

First, with reference to FIG. 14, an example of a system configurationaccording to the present embodiment is introduced. FIG. 14 is anexplanatory diagram for explaining an example of the systemconfiguration according to the present embodiment. Also, the systemconfiguration introduced herein is just an example and it is possible toapply a technique according to the present embodiment to various systemconfigurations available now and in the future.

As illustrated in FIG. 14, information provision systems 13, 17 and 19described below mainly include multiple information terminals CL and aserver apparatus SV. The information terminal CL is an example of adevice used by a user. As the information terminal CL, for example, amobile phone, a smart phone, a digital still camera, a digital videocamera, a personal computer, a table terminal, a car navigation system,a portable game device, health appliances (including a pedometer(registered trademark)) and medical equipment are assumed. Meanwhile, asthe server apparatus SV, for example, a home server and a cloudcomputing system are assumed.

Naturally, a system configuration to which a technique according to thepresent embodiment is applicable is not limited to the example in FIG.14, but, for convenience of explanation, an explanation is given with anassumption of the multiple information terminals CL and the serverapparatus SV which are connected by wired and/or wireless networks.Therefore, a configuration is assumed in which it is possible toexchange information between the information terminals CL and the serverapparatus SV. However, it is possible to employ a configuration suchthat, among various functions held by the information provision systems13, 17 and 19, functions to be held by the information terminals CL andfunctions to be held by the server apparatus SV are freely designed. Forexample, it is desirable to design it taking into account the computingpower and communication speed of the information terminals CL.

[2-2: Configuration Example #1 (Suggestion of Goal Attainment Level)]

First, configuration example #1 is explained. Configuration example #1relates to a system to provide, to a user, the attainment level withrespect to the user's goal/declaration acquired from input information.

(2-2-1: Details of System Configuration)

A system (i.e. information provision system 13) according toconfiguration example #1 is as illustrated in FIG. 15, for example. Asillustrated in FIG. 15, the information provision system 13 includes atext information acquiring unit 131, an experience extracting unit 132,a goal/declaration extracting unit 133, a goal/declaration checking unit134, a correspondence relationship storage unit 135 and agoal/declaration registering unit 136. Further, the informationprovision system 13 includes an attainment level storage unit 137, asensor information acquiring unit 138, an action pattern extracting unit139, an attainment level updating unit 140 and an attainment leveldisplaying unit 141.

Also, functions of the sensor information acquiring unit 138 and theaction pattern extracting unit 139 can be realized using a function ofthe action/situation analysis system 10 described above. Also, among theabove components held by the information provision system 13, it ispossible to freely design components whose functions are held by theinformation terminals CL and components whose functions are held by theserver apparatus SV. For example, it is desirable to design it takinginto account the computing power and communication speed of theinformation terminals CL.

The text information acquiring unit 131 acquires text information inputby a user. For example, the text information acquiring unit 131 maydenote an input device to input a text by the user or denote aninformation collection device to acquire text information from socialnetwork services or applications. Here, for convenience of explanation,an explanation is given with an assumption that the text informationacquiring unit 131 denotes an input unit such as a software keyboard.

The text information acquired by the text information acquiring unit 131is input in the experience extracting unit 132. At this time, theexperience extracting unit 132 may receive an input of the textinformation together with time information at the time of the input ofthe text information. When the text information is input, the experienceextracting unit 132 analyzes the input text information and extractsinformation related to user's experiences from the text information. Forexample, the information related to experiences denotes informationincluding an experienced event (such as an experience type), a place ofthe experience and the time of the experience.

Here, a functional configuration of the experience extracting unit 132is explained in detail with reference to FIG. 16. As illustrated in FIG.16, the experience extracting unit 132 mainly includes a type featureamount extracting unit 151, an experience type deciding unit 152 and anexperience type model storage unit 153. Further, the experienceextracting unit 132 includes a place feature amount extracting unit 154,an experience place extracting unit 155 and an experience place modelstorage unit 156. Further, the experience extracting unit 132 includes atime feature amount extracting unit 157, an experience time extractingunit 158 and an experience time model storage unit 159.

When the text information is input in the experience extracting unit132, the text information is input in the type feature amount extractingunit 151, the place feature amount extracting unit 154 and the timefeature amount extracting unit 157.

The type feature amount extracting unit 151 extracts a feature amountrelated to an experience type (hereinafter referred to as “type featureamount”) from the input text information. The type feature amountextracted by the type feature amount extracting unit 151 is input in theexperience type deciding unit 152. The experience type deciding unit 152decides an experience type from the input type feature amount, using alearning model stored in the experience type model storage unit 153.Subsequently, the decision result in the experience type deciding unit152 is input in the goal/declaration extracting unit 133.

Also, the place feature amount extracting unit 154 extracts a featureamount related to a place of the experience (hereinafter referred to as“place feature amount”) from the input text information. The placefeature amount extracted by the place feature amount extracting unit 154is input in the experience place deciding unit 155. The experience placedeciding unit 155 decides a place of the experience from the input placefeature amount, using a learning model stored in the experience placemodel storage unit 156. Subsequently, the decision result in theexperience place deciding unit 155 is input in the goal/declarationextracting unit 133.

Also, the time feature amount extracting unit 157 extracts a featureamount related to the time of the experience (hereinafter referred to as“time feature amount”) from the input text information. The time featureamount extracted by the time feature amount extracting unit 157 is inputin the experience time deciding unit 158. The experience time decidingunit 158 decides the time of the experience from the input time featureamount, using a learning model stored in the experience time modelstorage unit 159. Subsequently, the decision result in the experiencetime deciding unit 158 is input in the goal/declaration extracting unit133.

Here, with reference to FIG. 17, content of processing performed by theexperience extracting unit 132 is supplementarily explained using amusic experience as an example. FIG. 17 is an explanatory diagram forexplaining content of specific processing performed by the experienceextracting unit 132. Also, for convenience of explanation, although anexplanation is given using a music experience as an example, thetechnical scope of the present embodiment is not limited to this.

As illustrated in FIG. 17, in the case of a music experience, possibleexamples of the experience type include “listen to music (listen),”“watch a music video on TV/movie/DVD (watch),” “buy a track/CD (buy),”“participate in a live or concert (live)” and “sing/perform/compose asong (play).” The experience extracting unit 132 decides theseexperience types using the functions of the type feature amountextracting unit 151 and the experience type deciding unit 152.

For example, in the case of deciding an experience type of “listen,”first, the type feature amount extracting unit 151 extracts a typefeature amount related to the experience type of “listen” by a method ofmorpheme, n-gram or maximum substring. Next, the experience typedeciding unit 152 decides, from the type feature amount, whether itcorresponds to the experience type of “listen,” by a method such as SVMand logical regression. The decision result in the experience typedeciding unit 152 is output as information indicating the experiencetype. Similarly, decision results with respect to experience types of“watch,” “buy,” “live” and “play” are acquired.

Also, experience place extraction is realized by the functions of theplace feature amount extracting unit 154 and the experience placeextracting unit 155. First, the place feature amount extracting unit 154performs a morphological analysis for input text information and inputsthe result in the experience place extracting unit 155. Next, based onthe morphological analysis result, the experience place extracting unit155 extracts an experience place using a method such as CRF (ConditionalRandom Field). For example, the experience place extracting unit 155extracts an experience place (in the example in FIG. 19, extracts “Kyotostation”) as illustrated in FIG. 19, using a feature template asillustrated in FIG. 18.

Also, experience time extraction is realized by the functions of thetime feature amount extracting unit 157 and the experience timeextracting unit 158. Similar to the above experience place extraction,the experience time extraction is realized by a sequential labelingmethod using morphological analysis, CRF and so on. Also, as expressionof the experience time, for example, it is possible to use expression ofvarious units such as “present,” “past,” “future,” “morning,” “evening”and “night.” Information of the experience place and the experience timeacquired in this way is input together with the decision resultindicating the experience type, in the goal/declaration extracting unit133. Here, there is a case where part or all of the experience type, theexperience place and the experience type are not necessarily acquired.

FIG. 15 is referred to again. When the information of the experiencetype, the experience place and the experience type is acquired, thegoal/declaration extracting unit 133 decides whether the textinformation includes information related to goals/declaration, using theinformation of the experience type and the experience time. For example,as illustrated in FIG. 20, in a case where the experience type is “diet”and the experience time is “future,” the goal/declaration extractingunit 133 decides that text information corresponding to these resultsincludes the goal/declaration. Meanwhile, even when the experience typeis “diet,” in a case where the experience time is “past,” thegoal/declaration extracting unit 133 decides that text informationcorresponding to these results does not include the goal/declaration.

That is, in a case where the experience type corresponds to thegoal/declaration and the experience time is future, the goal/declarationextracting unit 133 decides that text information corresponding to theseresults includes the goal/declaration. Subsequently, thegoal/declaration extracting unit 133 extracts the experience typeacquired from the text information decided to include thegoal/declaration, as the goal/declaration. Subsequently, information ofthe goal/declaration extracted by the goal/declaration extracting unit133 is input in the goal/declaration checking unit 134. When theinformation of the goal/declaration is input, the goal/declarationchecking unit 134 refers to the correspondence relationship storage unit135, specifies one or multiple action patterns related to the inputgoal/declaration and extracts each specified action pattern.

Here, in the above explanation, although the goal/declaration checkingunit 134 specifies one or multiple action patterns with respect to theinformation of the goal/declaration after the information of thegoal/declaration is input, an applicable scope of the techniqueaccording to the present embodiment is not limited to this.

For example, all action patterns that can be acquired in advance may berecognized regardless of whether they are related to a goal/declaration,and the recognition results may be stored in a database. In this case,when a goal/declaration is input, data of an action pattern associatedwith the input goal/declaration may be referred to from the databasestoring all action pattern recognition results.

For example, as illustrated in FIG. 21, the correspondence relationshipstorage unit 135 stores a database showing correspondence relationshipsbetween goals/declaration and action patterns. Also, in the example inFIG. 21, the contribution level is associated with each of thecombinations between goals/declaration and action patterns. For example,in a case where dieting is the goal/declaration, actions such as“walking” and “running” are effective for the dieting but actions suchas “getting in a car” and “taking a train” are not effective for thedieting. From such a viewpoint, in the example in FIG. 21, thecontribution level is associated with each of combinations betweengoals/declaration and action patterns.

The goal/declaration checking unit 134 inputs information of agoal/declaration and information of an action pattern associated withthe goal/declaration in the goal/declaration registering unit 136. Whenthe information of the goal/declaration and the information of theaction pattern associated with the goal/declaration are input, thegoal/declaration registering unit 136 registers the inputgoal/declaration and action pattern in the attainment level storage unit137. When the goal/declaration is registered in this way, calculation ofthe attainment level with respect to the registered goal/declaration andprovision of information with respect to the attainment level start.Also, the attainment level is calculated according to an action patternevery day, and information of the attainment level with respect to thegoal/declaration is provided to a user in real time.

FIG. 15 is referred to again. Action pattern detection is realized bythe functions of the sensor information acquiring unit 138 and theaction pattern extracting unit 139. First, the sensor informationacquiring unit 138 acquires sensor information from a motion sensor,position sensor or the like. The sensor information acquired by thesensor information acquiring unit 138 is input in the action patternextracting unit 139. The action pattern extracting unit 139 extracts anaction pattern from the input sensor information. Information of theaction pattern extracted by the action pattern extracting unit 139 isinput in the attainment level updating unit 140. Also, as an actionpattern extraction method, it is possible to apply the same method asthe above action pattern extraction method by the action/situationanalysis system 10.

When the information of the action pattern is input, the attainmentlevel updating unit 140 refers to information related togoals/declaration registered in the attainment level storage unit 137,and decides whether the action pattern indicated by the inputinformation corresponds to an action pattern associated with thegoal/declaration. In a case where it corresponds to the action patternassociated with the action pattern associated with the goal/declaration,the attainment level storage unit 137 recognizes an attainment effect(for example, see FIG. 21) associated with a combination of thegoal/declaration and the input action pattern. Next, the attainmentlevel storage unit 137 calculates the current attainment level based onan update value of the attainment level associated with the attainmenteffect, and stores it in the attainment level storage unit 137.

For example, regarding a case where the goal/declaration is “dieting,”attainment effect “nothing” equals to −5 points, attainment effect “low”equals to +5 points, attainment effect “medium” equals to +15 points andattainment effect “high” equals to +30 points, it is specificallyconsidered with reference to FIG. 22. First, in a case where the usertakes a train for one hour, since an action pattern “taking a train(attainment effect “nothing”)” is detected, the attainment levelupdating unit 140 sets the current attainment level to “−5 points.”Next, in a case where the user walks for ten minutes, since an actionpattern “walking (attainment effect “medium”)” is detected, theattainment level updating unit 140 adds 15 points to the previousattainment level and updates the current attainment level to “10points.” Thus, the attainment level is updated based on an actionpattern.

FIG. 15 is referred to again. As described above, the attainment levelper goal/declaration stored in the attainment level storage unit 137 isupdated in real time. The attainment level per goal/declaration storedin the attainment level storage unit 137 is read by the attainment leveldisplaying unit 141 and presented to the user. For example, asillustrated in display examples #1 and #2 in FIG. 22, the attainmentlevel displaying unit 141 displays an object indicating an actionpattern taken by the user, together with a value of the attainment levelupdated according to the action. Display example #1 shows that, sincethe user takes an action “running,” the attainment level is increasedand the updated attainment level is set to 35.

The configuration of the information provision system 13 according toconfiguration example #1 has been explained above. Here, in the aboveexplanation, although an explanation has been given along a flow ofprocessing to analyze text information first and analyze sensorinformation later, the order of analysis processing may be reverse.Also, a report method of the attainment level may be audio guidanceinstead of screen display or may be an expression method by vibration orblinking. For example, there is a possible configuration in which theintensity of vibration changes according to the attainment level or theblinking speed or brightness changes. Such alternation naturally belongsto the technical scope of the present embodiment too.

(2-2-2: Flow of Processing)

Next, with reference to FIG. 23, a flow of processing performed by theinformation provision system 13 is explained. FIG. 23 is an explanatorydiagram for explaining a flow of processing performed by the informationprovision system 13. Also, the order of part of processing stepsillustrated in FIG. 23 may be changed. For example, the order of aprocessing step related to text information analysis and the order of aprocessing step related to sensor information analysis may be switched.

As illustrated in FIG. 23, first, the information provision system 13acquires text information by the function of the text informationacquiring unit 131 (S101). Next, the information provision system 13extracts information related to experiences, from the text information,by the function of the experience extracting unit 132 (S102). Examplesof the information related to experiences include an experience type, anexperience place and experience time. Next, the information provisionsystem 13 extracts information related to goals/declaration from theinformation related to experiences, by the function of thegoal/declaration extracting unit 133 (S103).

Next, the information provision system 13 extracts an action patterncorresponding to the goal/declaration extracted in step S103, by thefunction of the goal/declaration checking unit 134 (S104). Next, theinformation provision system 13 registers a goal/declaration for whichthe attainment level is calculated and an action pattern correspondingto the goal/declaration, in the attainment level storage unit 137, bythe function of the goal/declaration registering unit 136 (S105).

Meanwhile, the information provision system 13 acquires sensorinformation by the function of the sensor information acquiring unit 138(S106). Next, the information provision system 13 extracts an actionpattern from the sensor information by the function of the actionpattern extracting unit 139 (S107). Next, by the function of theattainment level updating unit 140, the information provision system 13recognizes an attainment effect of a goal/declaration corresponding tothe action pattern extracted in step S107 and calculates the currentattainment level based on the recognized attainment effect (S108). Next,the information provision system 13 displays the attainment level of thegoal/declaration by the function of the attainment level displaying unit141 (S109) and finishes a series of processing.

The flow of processing performed by the information provision system 13has been explained above.

(2-2-3: Example of Screen Display)

An attainment level display method is supplementarily explained below.

As an attainment level display method, there are methods such as displayexamples #1 and #2 in FIG. 22. That is, as an example, there is provideda method of displaying an object corresponding to the current actionpattern and displaying the attainment level acquired as a result oftaking the current action pattern. Especially, in the example in FIG.22, whether the attainment level is increased by the current actionpattern or the attainment level is decreased by the current actionpattern is indicated by an arrow, which causes an effect with respect toa goal/declaration to be identified at first sight. By such display, itis possible to encourage the user to take an action pattern leading to agood effect. Also, since the effect is reflected to a numerical value ofthe attainment level in real time, an effect of keeping motivation ofthe user tackling the goal/declaration can be expected.

(2-2-4: Alternation Example (Application to Animals))

By the way, an explanation has been given to the technique for humanaction patterns. However, the technique according to configurationexample #1 is applicable to other animals than human beings. Forexample, by wearing a sensor on a collar of a pet such as a dog and acat and inputting a goal/declaration of the pet as text information byan animal guardian, it is possible to acquire the goal attainment levelof the pet. For example, it is possible to acquire data such as theintensity of pet's activity in a place or time zone on which the animalguardian does not keep an eye. By analyzing such data and managing thepet's health, it is possible to produce an effect of preventing pet'sdisease or the like.

The technique according to configuration example #1 has been explainedabove. According to the technique according to above configurationexample #1, it is possible to present an attainment state related touser's declaration from the matching state of the user's declarationacquired from input information and an estimated action pattern.

Application Example

As described above, when the technique according to configurationexample #1 is applied, it is possible to acquire the attainment levelbased on user's goal/declaration and action pattern. Therefore, it ispossible to realize a display method of graphing this attainment leveland displaying it to the user or a display method of displaying thedegree of effort for the goal/declaration depending on whether theattainment level tends to increase for attainment of thegoal/declaration or the attainment level tends to decrease forattainment of the goal/declaration. Further, in a situation in which itis difficult to attain the goal/declaration (e.g. in a situation inwhich the attainment level is extremely low (such as a case where it isbelow a threshold)), it is possible to realize a display method of:presenting a representative example (with high frequency) or histogramof action patterns which are a cause of decreasing the attainment level;and presenting the cause of the difficult situation to the user.Further, by presenting an action pattern having an opposite tendency tothe representative example of action patterns which are a cause ofdecreasing the attainment level, it is possible to realize a displaymethod in which advice is given to the user to attain thegoal/declaration. By applying such a display method, it is possible todirectly or indirectly support the user to attain the goal/declaration.

[2-3: Configuration Example #2 (Display of Detailed Action)]

Next, configuration example #2 is explained. Configuration example #2relates to a system of adding information related to user's experiencesacquired from input information to action pattern information andproviding the result.

(2-3-1: Details of System Configuration)

A system (i.e. information provision system 17) according toconfiguration example #2 is as illustrated in FIG. 24, for example. Asillustrated in FIG. 24, the information provision system 17 includes atext information acquiring unit 171, an experience extracting unit 172,an extraction result storage unit 173, a sensor information acquiringunit 174, an action pattern extracting unit 175, an action/experiencechecking unit 176, a correspondence relationship storage unit 177, anadditional experience searching unit 178 and anaction/additional-experience displaying unit 179.

Also, functions of the sensor information acquiring unit 174 and theaction pattern extracting unit 175 can be realized using the function ofthe above action/situation analysis system 10. Also, among the abovecomponents held by the information provision system 17, it is possibleto freely design components whose functions are held by the informationterminals CL and components whose functions are held by the serverapparatus SV. For example, it is desirable to design it taking intoaccount the computing power and communication speed of the informationterminals CL.

The text information acquiring unit 171 acquires text information inputby the user. For example, the text information acquiring unit 171 maydenote an input device to input a text by the user or denote aninformation collection device to acquire text information from socialnetwork services or applications. Here, for convenience of explanation,an explanation is given with an assumption that the text informationacquiring unit 171 denotes an input unit such as a software keyboard.

The text information acquired by the text information acquiring unit 171is input in the experience extracting unit 172. At this time, theexperience extracting unit 172 may receive an input of the textinformation together with time information at the time of the input ofthe text information. When the text information is input, the experienceextracting unit 172 analyzes the input text information and extractsinformation related to user's experiences from the text information. Forexample, the information related to experiences denotes informationincluding an experienced event (such as an experience type), a place ofthe experience and the time of the experience. Also, the function of theexperience extracting unit 172 is substantially the same as the functionof the experience extracting unit 132 according to configuration example#1. The experience-related information extracted from the experienceextracting unit 172 is stored in the extraction result storage unit 173.

Meanwhile, the sensor information acquiring unit 174 acquires sensorinformation from a motion sensor, position sensor or the like. Thesensor information acquired by the sensor information acquiring unit 174is input in the action pattern extracting unit 175. The action patternextracting unit 175 extracts an action pattern from the input sensorinformation. Information of the action pattern extracted by the actionpattern extracting unit 175 is input in the action/experience checkingunit 176. Also, as an action pattern extraction method, it is possibleto apply the same method as the above action pattern extraction methodby the action/situation analysis system 10.

When the information of the action pattern is input, theaction/experience checking unit 176 refers to correspondencerelationships between action patterns and experiences, which are storedin the correspondence relationship storage unit 177, and extracts anexperience corresponding to the action pattern indicated by the inputinformation. For example, as illustrated in FIG. 25, the correspondencerelationship storage unit 177 stores experiences in association withaction patterns. As described above, an action pattern is acquired fromsensor information. Meanwhile, experience information is acquired fromtext information. A method of acquiring the action pattern and theexperience information is substantially the same as above configurationexample #1.

Information of the experience extracted by the action/experiencechecking unit 176 and information of the action pattern corresponding tothe experience are input in the additional experience searching unit178. When the experience information is input, the additional experiencesearching unit 178 refers to the extraction result storage unit 173 andsearches the same experience as the experience indicated by the inputinformation. As a result of the search, when the same experience as theexperience indicated by the input information is detected, theadditional experience searching unit 178 extracts text informationcorresponding to the detected experience and information related to theexperience (such as an experience type, experience place, experiencetime and experience target). For example, by the additional experiencesearching unit 178, it is possible to acquire information related toexperiences as illustrated in FIG. 26.

The search result in the additional experience searching unit 178 isinput in the action/additional-experience displaying unit 179. When thesearch result is input, the action/additional-experience displaying unit179 displays information related to the experience based on the inputsearch result. For example, as illustrated in FIG. 27, theaction/additional-experience displaying unit 179 displays informationrelated to action patterns and experiences. FIG. 27 illustrates the caseof displaying action pattern information acquired at the time of usingonly sensor information, together with the case of displaying actionpattern information and experience information acquired at the time ofusing sensor information and text information. When the text informationis used in addition to the sensor information, since it is possible toacquire detailed information related to experiences as illustrated inFIG. 27, it is possible to display the detailed information.

In the case of case #1, it is possible to display only an objectcorresponding to action pattern “walking” only by sensor information,but, when text information is additionally used, it is possible todisplay an object related to a “dog” which is an action target. In thecase of case #2, although it is possible to display only an objectcorresponding to action pattern “running” only by sensor information,but, when text information is additionally used, it is possible todisplay an object related to a “shrine” which is an experience target.

Further, in the case of case #3,although it is possible to display onlyan object corresponding to action pattern “getting in a car” only bysensor information, but, when text information is additionally used, itis possible to display an object related to experience type“conversation” and experience place “car.” Also, although a method ofadditionally using text information has been illustrated, since it ispossible to specify experience type “conversation” and experience place“car” even by using sound information, it is possible to realize thesimilar detailed display by additionally using sound information. Also,when a sound recognition technique is used, since it is possible toconvert sound signals into text information, it is possible to realizethe detailed display as illustrated in FIG. 27 by the similar method.

The configuration of the information provision system 17 according toconfiguration example #2 has been explained above. Here, in the aboveexplanation, although an explanation has been given along a flow ofprocessing to analyze text information first and analyze sensorinformation later, the order of analysis processing may be reverse.Also, a report method of detailed information may be audio guidanceinstead of screen display. Such alternation naturally belongs to thetechnical scope of the present embodiment too.

(2-3-2: Flow of Processing)

Next, with reference to FIG. 28, a flow of processing performed by theinformation provision system 17 is explained. FIG. 28 is an explanatorydiagram for explaining a flow of processing performed by the informationprovision system 17. Also, the order of part of the processing stepsillustrated in FIG. 28 may be changed. For example, the order of aprocessing step related to text information analysis and the order of aprocessing step related to sensor information analysis may be switched.

As illustrated in FIG. 28, first, the information provision system 17acquires text information by the function of the text informationacquiring unit 171 (S111). Next, the information provision system 17extracts information related to experiences, from the text information,by the function of the experience extracting unit 172 (S112). Next, theinformation provision system 17 acquires sensor information by thefunction of the sensor information acquiring unit 174 (S113). Next, theinformation provision system 17 extracts an action pattern from thesensor information by the function of the action pattern extracting unit175 (S114).

Next, the information provision system 17 checks the action patternextracted in step S114 against the experiences by the function of theaction/experience checking unit 176, and extracts information of anexperience corresponding to the action pattern (S115). Next, theinformation provision system 17 extracts experience-related informationcorresponding to the experience extracted in step S115, from informationrelated to the experiences extracted in step S112, by the function ofthe additional experience searching unit 178 (S116). Next, theinformation provision system 17 displays information corresponding tothe action pattern extracted from the sensor information, together withinformation corresponding to the experience-related informationextracted in step S116, by the function of theaction/additional-experience displaying unit 179 (S117), and finishes aseries of processing.

The flow of processing performed by the information provision system 17has been explained above.

(2-3-3: Example of Screen Display)

In the following, a display method of detailed information issupplementarily explained.

As the detailed information display method, there are methods such ascases #1 to #3 in FIG. 27. That is, as an example, there is provided amethod of displaying information of an action pattern detected fromsensor information, and, in a case where experience-related informationwith respect to an experience corresponding to the action pattern isacquired from text information, additionally displaying theexperience-related information. Also, there is a possible method ofadditionally displaying time information (“in five minutes”) as detaileddisplay like case #2 or additionally displaying conversation content ofa fellow passenger as detailed display like case #3. If such detaileddisplay is possible, it is possible to report a state more accurately.

The technique according to configuration example #2 has been explainedabove.

[2-4: Configuration Example #3: (Decision of Ordinary Action orExtraordinary Action)]

Next, configuration example #3 is explained. Configuration example #3relates to a system of deciding an extraordinary action or experienceamong user's experiences acquired from input information such as actionpattern information and text information, and providing it to the user.

(2-4-1: Details of System Configuration)

A system (information provision system 19) according to configurationexample #3 is as illustrated in FIG. 29, for example. As illustrated inFIG. 29, the information provision system 19 includes a text informationacquiring unit 191, an experience extracting unit 192, an extractionresult storage unit 193, a sensor information acquiring unit 194, anaction pattern extracting unit 195, an action/experience checking unit196, a correspondence relationship storage unit 197, an extraordinaryaction deciding unit 198 and an extraordinary action displaying unit199.

Also, functions of the sensor information acquiring unit 194 and theaction pattern extracting unit 195 can be realized using the function ofthe above action/situation analysis system 10. Also, among the abovecomponents held by the information provision system 19, it is possibleto freely design components whose functions are held by the informationterminals CL and components whose functions are held by the serverapparatus SV. For example, it is desirable to design it taking intoaccount the computing power and communication speed of the informationterminals CL.

The text information acquiring unit 191 acquires text information inputby the user. For example, the text information acquiring unit 191 maydenote an input device to input a text by the user or denote aninformation collection device to acquire text information from socialnetwork services or applications. Here, for convenience of explanation,an explanation is given with an assumption that the text informationacquiring unit 191 denotes an input unit such as a software keyboard.

The text information acquired by the text information acquiring unit 191is input in the experience extracting unit 192. At this time, theexperience extracting unit 192 may receive an input of the textinformation together with time information at the time of the input ofthe text information. When the text information is input, the experienceextracting unit 192 analyzes the input text information and extractsinformation related to user's experiences from the text information. Forexample, the information related to experiences denotes informationincluding an experienced event (such as an experience type), a place ofthe experience and the time of the experience. Also, the function of theexperience extracting unit 192 is substantially the same as the functionof the experience extracting unit 132 according to configuration example#1. The experience-related information extracted from the experienceextracting unit 192 is stored in the extraction result storage unit 193.

Meanwhile, the sensor information acquiring unit 194 acquires sensorinformation from a motion sensor, position sensor or the like. Thesensor information acquired by the sensor information acquiring unit 194is input in the action pattern extracting unit 195. The action patternextracting unit 195 extracts an action pattern from the input sensorinformation. Information of the action pattern extracted by the actionpattern extracting unit 195 is input in the action/experience checkingunit 196. Also, as an action pattern extraction method, it is possibleto apply the same method as the above action pattern extraction methodby the action/situation analysis system 10.

When the information of the action pattern is input, theaction/experience checking unit 196 refers to correspondencerelationships between action patterns and experiences, which are storedin the correspondence relationship storage unit 197, and extracts anexperience corresponding to the action pattern indicated by the inputinformation. For example, as illustrated in FIG. 25, the correspondencerelationship storage unit 197 stores experiences in association withaction patterns. As described above, an action pattern is acquired fromsensor information. Meanwhile, experience information is acquired fromtext information. A method of acquiring the action pattern and theexperience information is substantially the same as above configurationexample #1.

Information of the experience extracted by the action/experiencechecking unit 196 and information of the action pattern corresponding tothe experience are input in the extraordinary action deciding unit 198.When the action pattern information is input, the extraordinary actiondeciding unit 198 decides whether the input action pattern informationindicates an extraordinary action. Also, when the experience informationis input, the extraordinary action deciding unit 198 decides whether theinput experience information indicates an extraordinary experience.

For example, the extraordinary action deciding unit 198 decides anextraordinary action and an extraordinary experience based onextraordinary conditions as illustrated in FIG. 30.

In the example in FIG. 30, in the case of (extraordinary #1), theextraordinary action deciding unit 198 decides whether a time zoneabnormity occurs in an action pattern extracted from sensor information.That is, in a case where an action of a certain type is extracted in atime zone different from a time zone in which it is ordinarily extractedor in a case where it is not extracted in all time zones, theextraordinary action deciding unit 198 decides the action as anextraordinary action. To be more specific, regarding a user for which a“walking” action is ordinarily extracted in the morning and evening, ina case where the “walking” action is extracted at midnight, the“walking” action at midnight is decided as an extraordinary action.

Also, in the case of (extraordinary #2), the extraordinary actiondeciding unit 198 decides whether a type abnormity occurs in an actionpattern extracted from sensor information. That is, in a case where anaction of a different type from the type of an action that is ordinarilyextracted is extracted in a certain time zone, the extraordinary actiondeciding unit 198 decides the action as an extraordinary action. To bemore specific, regarding a user for which a “walking” action or a“train” action is ordinarily extracted in the morning, in a case where a“running” action or a “bicycle” action is extracted, the “running”action and the “bicycle” action are decided as an extraordinary action.

Also, in the case of (extraordinary #3), the extraordinary actiondeciding unit 198 decides whether a time zone abnormity occurs in anexperience extracted from text information. That is, in a case where anexperience of a certain type is extracted in a time zone different froma time zone in which it is ordinarily extracted or in a case where it isnot extracted in all time zones, the extraordinary action deciding unit198 decides the experience as an extraordinary action. To be morespecific, regarding a user for which an “eating” experience is extractedin the morning, afternoon and evening, in a case where the “eating”experience is extracted at midnight or in a case where the “eating”experience is not extracted in the afternoon, the correspondingexperience is decided as an extraordinary action.

Also, in the case of (extraordinary #4), the extraordinary actiondeciding unit 198 decides whether a type abnormity occurs in anexperience extracted from text information. That is, in a case where anexperience of a different type from the type of an experience that isordinarily extracted is extracted in a certain time zone, theextraordinary action deciding unit 198 decides the experience as anextraordinary action. To be more specific, regarding a user for which an“eating” experience is ordinarily extracted in the afternoon, in a casewhere a “running” experience is detected in the afternoon, the “running”experience is decided as an extraordinary action.

FIG. 29 is referred to again. As described above, an extraordinaryaction decision result by the extraordinary action deciding unit 198 isinput in the extraordinary action displaying unit 199. The extraordinaryaction displaying unit 199 highlights an object or text corresponding toan extraordinary action or displays a new object indicating anextraordinary action.

The configuration of the information provision system 19 according toconfiguration example #3 has been explained above. Here, in the aboveexplanation, although an explanation has been given along a flow ofprocessing to analyze text information first and analyze sensorinformation later, the order of analysis processing may be reverse.Also, although an explanation has been given using a processing exampleto decide the ordinary/extraordinary, it is possible to similarly form asystem to decide a positive action or negative action. Such alternationnaturally belongs to the technical scope of the present embodiment too.

(2-4-2: Application Example)

Although extraordinary action decision logic is specifically illustratedin FIG. 30, by deciding an ordinary action or extraordinary action inthis way, it is possible to estimate user's health status or detailedstate. For example, in the case of (extraordinary #1) illustrated inFIG. 30, since a midnight action that is not ordinarily extracted isextracted, insomnia may be estimated. Although conditions are simplifiedfor convenience of explanation in the example in FIG. 30, for example,if a “catnap” action is extracted during the day, the late hours or liferhythm disturbance is estimated as the reason. By further addingconditions, it can be used to the diagnosis of health status such asinsomnia.

Similarly, in the example of (extraordinary #2), as a reason of theextraordinary action, a challenge to dieting during commute time may beestimated. Also, in the example of (extraordinary #3), busyness at workis simply estimated as a reason of the extraordinary action. Further, inthe example of (extraordinary #4), as a reason of the extraordinaryaction, a challenge to dieting by skipping lunch may be estimated. Bycombining extraordinary actions, it is possible to improve the reasonestimation accuracy. Also, from a history or statistical result ofextraordinary actions based on sensor information and text information,it is effective to improve extraordinary conditions or form a reasonestimation algorithm. Thus, the technique according to the presentembodiment is variously applicable.

The technique according to configuration example #3 has been explainedabove.

[2-5: Regarding Combination of Configuration Examples]

It is possible to arbitrarily combine the techniques according to aboveconfiguration examples #1 to #3. Since the technique of extractingexperience-related information from text information and the techniqueof extracting an action pattern from sensor information are common, itis possible to arbitrarily combine part of all of configuration examples#1 to #3 by connecting other function blocks in series or parallel.Also, in the case of combining multiple configuration examples, bymaking a design to share a function block having a common function, aneffect of processing load reduction or memory usage reduction isestimated. Such a combination configuration naturally belongs to thetechnical scope of the present embodiment too.

3: EXAMPLE HARDWARE CONFIGURATION

Functions of each constituent included in the action/situation analysissystem 10, the information provision systems 13, 17, and 19, theinformation terminal CL, and the server apparatus SV described above canbe realized by using, for example, the hardware configuration of theinformation processing apparatus shown in FIG. 31. That is, thefunctions of each constituent can be realized by controlling thehardware shown in FIG. 31 using a computer program. Additionally, themode of this hardware is arbitrary, and may be a personal computer, amobile information terminal such as a mobile phone, a PHS or a PDA, agame machine, or various types of information appliances. Moreover, thePHS is an abbreviation for Personal Handy-phone System. Also, the PDA isan abbreviation for Personal Digital Assistant.

As shown in FIG. 31, this hardware mainly includes a CPU 902, a ROM 904,a RAM 906, a host bus 908, and a bridge 910. Furthermore, this hardwareincludes an external bus 912, an interface 914, an input unit 916, anoutput unit 918, a storage unit 920, a drive 922, a connection port 924,and a communication unit 926. Moreover, the CPU is an abbreviation forCentral Processing Unit. Also, the ROM is an abbreviation for Read OnlyMemory. Furthermore, the RAM is an abbreviation for Random AccessMemory.

The CPU 902 functions as an arithmetic processing unit or a controlunit, for example, and controls entire operation or a part of theoperation of each structural element based on various programs recordedon the ROM 904, the RAM 906, the storage unit 920, or a removalrecording medium 928. The ROM 904 is a mechanism for storing, forexample, a program to be loaded on the CPU 902 or data or the like usedin an arithmetic operation. The RAM 906 temporarily or perpetuallystores, for example, a program to be loaded on the CPU 902 or variousparameters or the like arbitrarily changed in execution of the program.

These structural elements are connected to each other by, for example,the host bus 908 capable of performing high-speed data transmission. Forits part, the host bus 908 is connected through the bridge 910 to theexternal bus 912 whose data transmission speed is relatively low, forexample. Furthermore, the input unit 916 is, for example, a mouse, akeyboard, a touch panel, a button, a switch, or a lever. Also, the inputunit 916 may be a remote control that can transmit a control signal byusing an infrared ray or other radio waves.

The output unit 918 is, for example, a display device such as a CRT, anLCD, a PDP or an ELD, an audio output device such as a speaker orheadphones, a printer, a mobile phone, or a facsimile, that can visuallyor auditorily notify a user of acquired information. Moreover, the CRTis an abbreviation for Cathode Ray Tube. The LCD is an abbreviation forLiquid Crystal Display. The PDP is an abbreviation for Plasma DisplayPanel. Also, the ELD is an abbreviation for Electro-LuminescenceDisplay.

The storage unit 920 is a device for storing various data. The storageunit 920 is, for example, a magnetic storage device such as a hard diskdrive (HDD), a semiconductor storage device, an optical storage device,or a magneto-optical storage device. The HDD is an abbreviation for HardDisk Drive.

The drive 922 is a device that reads information recorded on the removalrecording medium 928 such as a magnetic disk, an optical disk, amagneto-optical disk, or a semiconductor memory, or writes informationin the removal recording medium 928. The removal recording medium 928is, for example, a DVD medium, a Blu-ray medium, an HD-DVD medium,various types of semiconductor storage media, or the like. Of course,the removal recording medium 928 may be, for example, an electronicdevice or an IC card on which a non-contact IC chip is mounted. The ICis an abbreviation for Integrated Circuit.

The connection port 924 is a port such as an USB port, an IEEE1394 port,a SCSI, an RS-232C port, or a port for connecting an externallyconnected device 930 such as an optical audio terminal. The externallyconnected device 930 is, for example, a printer, a mobile music player,a digital camera, a digital video camera, or an IC recorder. Moreover,the USB is an abbreviation for Universal Serial Bus. Also, the SCSI isan abbreviation for Small Computer System Interface.

The communication unit 926 is a communication device to be connected toa network 932, and is, for example, a communication card for a wired orwireless LAN, Bluetooth (registered trademark), or WUSB, an opticalcommunication router, an ADSL router, or a modem for variouscommunication. The network 932 connected to the communication unit 926is configured from a wire-connected or wirelessly connected network, andis the Internet, a home-use LAN, infrared communication, visible lightcommunication, broadcasting, or satellite communication, for example.Moreover, the LAN is an abbreviation for Local Area Network. Also, theWUSB is an abbreviation for Wireless USB. Furthermore, the ADSL is anabbreviation for Asymmetric Digital Subscriber Line.

4: CONCLUSION

Finally, the technical idea of the present embodiment is simplysummarized. The technical idea described below is applicable to variousinformation processing apparatuses such as a PC, a mobile phone, aportable game device, a portable information terminal, an informationappliance and a car navigation.

Additionally, the present technology may also be configured as below.

1. An information processing apparatus including:

-   -   a matter extracting unit extracting a predetermined matter from        text information;    -   an action pattern specifying unit specifying one or multiple        action patterns associated with the predetermined matter;    -   an action extracting unit extracting each of the action patterns        associated with the predetermined matter, from sensor        information; and    -   a state analyzing unit generating state information indicating a        state related to the predetermined matter, based on each of the        action patterns extracted from the sensor information, using a        contribution level indicating a degree of contribution of each        of the action patterns to the predetermined matter, for a        combination of the predetermined matter and each of the action        patterns associated with the predetermined matter.

(2) The information processing apparatus according to (1), furtherincluding:

-   -   a state displaying unit displaying the state information        generated by the state analyzing unit.

(3) The information processing apparatus according to (1) or (2),further including:

-   -   a sensor information acquiring unit acquiring sensor information        detected by sensor mounted on a terminal apparatus held by a        user; and    -   a text information acquiring unit acquiring text information        input by the user,    -   wherein the matter extracting unit extracts the predetermined        matter from the text information acquired by the text        information acquiring unit, and    -   wherein the action extracting unit extracts the action patterns        from the sensor information acquired by the sensor information        acquiring unit.

(4) The information processing apparatus according to (2), wherein thestate displaying unit displays the state information on a display unitof a terminal apparatus held by a user.

(5) The information processing apparatus according to any one of (1) to(4),

-   -   wherein the state information includes at least one of        -   an attainment level indicating, in a case where the            predetermined matter is a matter desired to be attained, a            current attainment level with respect to the matter,        -   a current value related to a current state of the            predetermined matter, which is acquired from the action            patterns associated with the predetermined matter, and        -   a comparison value obtained by comparing the current value            related to the current state of the predetermined matter and            a past value related to a past state of the predetermined            matter, the comparison value being a value acquired from the            action patterns associated with the predetermined matter.

(6) The information processing apparatus according to any one of (3) to(5),

-   -   wherein, in a case where the predetermined matter is a matter        desired to be attained, the state information is an attainment        level indicating a current attainment level with respect to the        matter, and    -   wherein the information processing apparatus further includes:        -   an effort tendency reporting unit deciding whether an effort            is made to attain the matter desired to be attained,            according to an increase tendency or a decrease tendency of            the attainment level, and presenting a decision result to a            user.

(7) The information processing apparatus according to any one of (3) to(6),

-   -   wherein, in a case where the predetermined matter is a matter        desired to be attained the state information is an attainment        level indicating a current attainment level with respect to the        matter, and    -   wherein the information processing apparatus further includes:        -   a cause reporting unit presenting, to a user, an action            pattern of high frequency among action patters that have            previously provided negative contribution levels or a            histogram of the action patterns that have previously            provided negative contribution levels in a case where the            attainment level is below a predetermined threshold.

(8) The information processing apparatus according to any one of (3) to(7),

-   -   wherein, in a case where the predetermined matter is a matter        desired to be attained, the state information is an attainment        level indicating a current attainment level with respect to the        matter, and    -   wherein the information processing apparatus further includes:        -   an advice reporting unit presenting, to a user, an action            pattern having an opposite tendency to an action patter of            high frequency among action patterns that have previously            provided negative contribution levels in a case where the            attainment level is below a predetermined threshold.

(9) The information processing apparatus according to (1) or (2),further including:

-   -   a sensor information acquiring unit acquiring sensor information        detected by a sensor mounted on a terminal apparatus attached to        a mobile object; and    -   a text information acquiring unit acquiring text information        input by a user who manages the mobile object,    -   wherein the matter extracting unit extracts the predetermined        matter from the text information acquired by the text        information acquiring unit, and    -   wherein the action extracting unit extracts the action patterns        from the sensor information acquired by the sensor information        acquiring unit.

(10) An information processing method including:

-   -   extracting a predetermined matter from text information;        specifying one or multiple action patterns associated with the        predetermined matter;    -   extracting each of the action patterns associated with the        predetermined matter, from sensor information; and    -   generating state information indicating a state related to the        predetermined matter, based on each of the action patterns        extracted from the sensor information, using a contribution        level indicating a degree of contribution of each of the action        patterns to the predetermined matter, for a combination of the        predetermined matter and each of the action patterns associated        with the predetermined matter.

(11) A program for causing a computer to realize:

-   -   a matter extracting function of extracting a predetermined        matter from text information;    -   an action pattern specifying function of specifying one or        multiple action patterns associated with the predetermined        matter;    -   an action extracting function of extracting each of the action        patterns associated with the predetermined matter, from sensor        information; and    -   a state analyzing function of generating state information        indicating a state related to the matter, based on each of the        action patterns extracted from the sensor information, using a        contribution level indicating a degree of contribution of each        of the action patterns to the predetermined matter, for a        combination of the predetermined matter and each of the action        patterns associated with the predetermined matter.

(12) An information processing apparatus including:

-   -   an experience extracting unit extracting experience information        indicating a user experience from text information;    -   an action extracting unit extracting an action pattern from        sensor information;    -   a correspondence experience extracting unit extracting, based on        relationship information indicating a correspondence        relationship between the experience information and the action        pattern, experience information corresponding to the action        pattern extracted from the sensor information; and    -   a display controlling unit displaying information related to the        experience information extracted from the text information along        with information related to the experience information        corresponding to the action pattern.

(13) The information processing apparatus according to (12), wherein theexperience extracting unit extracts information of at least one of anexperience type, an experience place, an experience time and anexperience target, from the text information, as the experienceinformation.

(14) The information processing apparatus according to (12) or (13),wherein the display controlling unit displays the information related tothe experience information corresponding to the action pattern extractedfrom the sensor information, and, in a case where a user performs anoperation of detailed display, displays the information related to theexperience information extracted from the text information.

(15) The information processing apparatus according to any one of (12)to (14), further including:

-   -   a sensor information acquiring unit acquiring sensor information        detected by a sensor mounted on a terminal apparatus held by a        user; and    -   a text information acquiring unit acquiring text information        input by the user,    -   wherein the experience extracting unit extracts the experience        information from the text information acquired by the text        information acquiring unit, and    -   wherein the action extracting unit extracts the action pattern        from the sensor information acquired by the sensor information        acquiring unit.

(16) The information processing apparatus according to any one of (12)to (15), further including:

-   -   an extraordinary action deciding unit deciding whether the        action pattern extracted from the sensor information is        extraordinary.

(17) The information processing apparatus according to (16), wherein theextraordinary action deciding unit further decides whether theexperience information extracted from the text information isextraordinary.

(18) The information processing apparatus according to (16) or (17),wherein, in a case where the extraordinary action deciding unit decidesthat the experience information extracted from the text information isextraordinary, the display controlling unit highlights informationrelated to experience information corresponding to a result of thedecision.

(19) The information processing apparatus according to any one of (16)to (18), wherein the extraordinary action deciding unit decides anaction corresponding to experience information extracted in a time zonedifferent from a time zone that is ordinarily extracted, or an actioncorresponding to experience information that is not extracted in bothtime zones, as an extraordinary action.

(20) The information processing apparatus according to any one of (16)to (18), wherein the extraordinary action deciding unit decides anaction corresponding to experience information of a type different froma type of an experience that is ordinarily extracted, as anextraordinary action.

(21) An information processing method including:

-   -   extracting experience information indicating a user experience,        from text information;    -   extracting an action pattern from sensor information;    -   extracting, based on relationship information indicating a        correspondence relationship between the experience information        and the action pattern, experience information corresponding to        the action pattern extracted from the sensor information; and    -   displaying information related to the experience information        extracted from the text information along with information        related to the experience information corresponding to the        action pattern.

(21) A program causing a computer to realize:

-   -   an experience extracting function of extracting experience        information indicating a user experience, from text information;    -   an action extracting function of extracting an action pattern        from sensor information;    -   a correspondence experience extracting function of extracting,        based on relationship information indicating a correspondence        relationship between the experience information and the action        pattern, experience information corresponding to the action        pattern extracted from the sensor information; and    -   a display controlling function of displaying information related        to the experience information extracted from the text        information along with information related to the experience        information corresponding to the action pattern.

Although the preferred embodiments of the present disclosure have beendescribed in detail with reference to the appended drawings, the presentdisclosure is not limited thereto. It is obvious to those skilled in theart that various modifications or variations are possible insofar asthey are within the technical scope of the appended claims or theequivalents thereof. It should be understood that such modifications orvariations are also within the technical scope of the presentdisclosure.

What is claimed is:
 1. An information processing apparatus, comprising:circuitry configured to determine action data of a user based on motiondata received from a motion sensor, determine a goal or declarationbased on text information, the text information being obtained from atleast one of social media content and audio data and being associatedwith a type of the goal or declaration and a time of the goal ordeclaration, retrieve state information of the user generated based onthe action data, the type of the goal or declaration, and the time ofthe goal or declaration, and control a display of an attainment levelwith respect to the goal or declaration generated from the stateinformation, wherein the attainment level includes a current valuerelated to current state, a graph of the attainment level of the user,and a degree of effort for the goal or declaration.
 2. The informationprocessing apparatus according to claim 1, wherein the attainment levelfurther includes an object corresponding to the current value.
 3. Theinformation processing apparatus according to claim 1, wherein theattainment level further includes a comparison value obtained bycomparing the current value and a past value related to past stateinformation.
 4. The information processing apparatus according to claim1, wherein the action data is associated with a physical activityundertaken by the user.
 5. The information processing apparatusaccording to claim 1, wherein the attainment level further includes acomparison value between the current state and a state in a case wherethe goal or declaration is attained.
 6. An information processingmethod, comprising: determining action data of a user based on motiondata received from a motion sensor; determining, by circuitry of aninformation processing apparatus, a goal or declaration based on textinformation, the text information being obtained from at least one ofsocial media content and audio data and being associated with a type ofthe goal or declaration and a time of the goal or declaration;retrieving, by the circuitry, state information of the user generatedbased on the action data, the type of the goal or declaration, and thetime of the goal or declaration; and controlling a display of anattainment level with respect to the goal or declaration generated fromthe state information, wherein the attainment level includes a currentvalue related to a current state, a graph of the attainment level of theuser, and a degree of effort for the goal or declaration.
 7. Theinformation processing method according to claim 6, wherein theattainment level further includes an object corresponding to the currentvalue.
 8. The information processing method according to claim 6,wherein the attainment level further includes a comparison valueobtained by comparing the current value and a past value related to paststate information.
 9. The information processing method according toclaim 6, wherein the action data is associated with a physical activityundertaken by the user.
 10. The information processing method accordingto claim 6, wherein the attainment level further includes a comparisonvalue between the current state and a state in a case where the goal ordeclaration is attained.
 11. A non-transitory computer-readable mediumstoring program code which when executed by a computer cause thecomputer to perform an information processing method, the informationprocessing method comprising: determining action data of a user based onmotion data received from a motion sensor; determining a goal ordeclaration based on text information, the text information beingobtained from at least one of social media content and audio data andbeing associated with a type of the goal or declaration and a time ofthe goal or declaration; retrieving state information of the usergenerated based on the action data, the type of the goal or declaration,and the time of the goal or declaration; and controlling a display of anattainment level with respect to the goal or declaration generated fromthe state information, wherein the attainment level includes a currentvalue related to a current state, a graph of the attainment level of theuser, and a degree of effort for the goal or declaration.
 12. Thenon-transitory computer-readable medium according to claim 11, whereinthe attainment level further includes an object corresponding to thecurrent value.
 13. The non-transitory computer-readable medium accordingto claim 11, wherein the attainment level further includes a comparisonvalue obtained by comparing the current value and a past value relatedto past state information.
 14. The non-transitory computer-readablemedium according to claim 11, wherein the action data is associated witha physical activity undertaken by the user.
 15. The non-transitorycomputer-readable medium according to claim 11, wherein the attainmentlevel further includes a comparison value between a current state and astate in a case where the goal or declaration is attained.